Adaptive clinical trials in surgery: A scoping review of methodological and reporting quality

Importance Adaptive surgical trials are scarce, but adopting these methods may help elevate the quality of surgical research when large-scale RCTs are impractical. Objective Randomized-controlled trials (RCTs) are the gold standard for evidence-based healthcare. Despite an increase in the number of RCTs, the number of surgical trials remains unchanged. Adaptive clinical trials can streamline trial design and time to trial reporting. The advantages identified for ACTs may help to improve the quality of future surgical trials. We present a scoping review of the methodological and reporting quality of adaptive surgical trials. Evidence review We performed a search of Ovid, Web of Science, and Cochrane Collaboration for all adaptive surgical RCTs performed from database inception to October 12, 2023. We included any published trials that had at least one surgical arm. All review and abstraction were performed in duplicate. Risk of bias (RoB) was assessed using the RoB 2.0 instrument and reporting quality was evaluated using CONSORT ACE 2020. All results were analyzed using descriptive methods. Findings Of the 1338 studies identified, six trials met inclusion criteria. Trials were performed in cardiothoracic, oral, orthopedic, and urological surgery. The most common type of adaptive trial was group sequential design with pre-specified interim analyses planned for efficacy, futility, and/or sample size re-estimation. Two trials did use statistical simulations. Our risk of bias evaluation identified a high risk of bias in 50% of included trials. Reporting quality was heterogeneous regarding trial design and outcome assessment and details in relation to randomization and blinding concealment. Conclusion and relevance Surgical trialists should consider implementing adaptive components to help improve patient recruitment and reduce trial duration. Reporting of future adaptive trials must adhere to existing CONSORT ACE 2020 guidelines. Future research is needed to optimize standardization of adaptive methods across medicine and surgery.


Introduction
Randomized-controlled trials (RCTs) are essential for evaluating the effectiveness and safety of interventions in healthcare [1].Their importance is reflected in the literature: since 1965, over 39,000 RCTs have been published globally, with over 60% published in the last 20 years [2].In 2003, however, only 3.4% of studies published in leading journals were surgical RCTs [3].Despite a 50% increase in the number of published surgical trials between 1999 and 2009, this number has remained stable over the past decade [4].Surgical trials suffer from a high rate of discontinuation and nonpublication rates often due to slow patient recruitment [5].A systematic review of surgical trials published from 2008 to 2020 highlighted several methodological concerns with surgical RCTs, including small sample sizes, a focus on minor clinical outcomes, moderate-to-high bias, and inconsistent usage of blinding and expertise-based randomization [3,6].
In medicine, at least 50% of adopted interventions are derived from RCTs, yet fewer than 25% of surgical interventions are based on evidence derived from RCTs [3].Adherence to clinical trial methodological standards in surgery is often impacted by high costs, feasibility issues, between-group crossover, and poor patient adherence [7,8].As a consequence, many surgical innovations have been adopted based upon non-scientific practices and small-scale, poorly controlled observational studies [6].Randomized studies in surgery, however, have historically led to an effective discarding of unnecessary surgical procedures [9].
The conventional randomized trial design with a large sample size remains the gold-standard approach for comparing medical interventions.Classically, RCTs adhere to a fixed study protocol and culminate in a pre-defined final analysis.Adaptive clinical trials, however, involve flexible adjustments to the study protocol based upon pre-specified interim analyses, which can permit sample size re-calculations, adding or dropping treatment arms, and/or stopping the trial for futility or lack of efficacy (Table 1).Adaptive trials are gaining popularity in drug development and other medical disciplines, as demonstrated by the TAILoR, 18-F PET, and STAMPEDE trials [10][11][12].Benefits of trial adaptability include cost reduction, decreased probability of assigning patients to an ineffective treatment arm, and expedited trial completion [13].Adaptive trials were useful during the peak of COVID-19 to accelerate the comparison of anti-viral therapies [14].Adaptive methodology, however, has been less adopted in evidence-based surgery.Given the challenges surgical researchers face in implementing conventional RCTs, adaptive trials may represent a high-quality alternative that allows surgical researchers to retain the benefits of randomization whilst minimizing costs and permitting protocol adjustments to maximize trial feasibility, adherence, and validity.The goal of this study was to perform a scoping review to characterize the methodological and reporting quality of adaptive study designs in surgical trials.These findings will define the current landscape of adaptive surgical trial quality so that they can be optimized and applied to future surgical populations.

Study design
We performed a scoping review of all prospective randomized trials that have employed adaptive designs within any surgical discipline.Surgical disciplines for the purposes of this study included cardiothoracic surgery, general surgery, gynecological surgery, orthopedic surgery, ophthalmology, otolaryngology, neurosurgery, plastic surgery, and urological surgery.We included all studies that compared at least one surgical arm.We also included studies that compared at least one surgical arm to a non-surgical interventional procedure.We registered this review in Open Science Framework (DOI: 10.17605/OSF.IO/2GDC5).Due to the nature of this study, institutional review board (IRB) approval was not required.This review was performed in accordance with PRISMA Scoping Review guidelines [15].

Search strategy
We performed a library citation search of Medline (Ovid), EMBASE (Ovid), Web of Science, Cochrane Collaboration, and CENTRAL databases from inception to October 14th, 2023 (S1 File).The search strategy was finalized by a librarian specialist.We performed a search of preprint databases to identify any relevant drafted manuscripts or ongoing clinical trials.We also performed a search of clinicaltrials.govusing the keywords "adaptive/Bayesian", "clinical trial/ trial", and "surgery/surgical" to evaluate for any active surgical trials meeting our eligibility criteria.We also used keywords for each included surgical discipline.We screened the first 10 pages of relevant results of clinicaltrials.govand pre-publication databases.Additional studies were identified through reference list searches of included articles.All duplicates were removed, and citations were managed using Covidence software (Melbourne, Victoria, Australia) [16].

Study selection
Any adaptive trial investigating one or more surgical or procedural interventions was included.Eligible trials must have included one of the following adaptive designs: Bayesian, frequentist, sample re-estimation, group sequential, multi-arm multi-stage (MAMS), seamless, continual reassessment, population enrichment, adaptive randomization, and/or adaptive dose-ranging.Any protocols for adaptive surgical trials were collated for the discussion, but not included in the final article synthesis.We excluded any trials that did not include at least one surgical arm, as well as any trials evaluating perioperative medications or non-surgical interventions provided in a surgical setting.Abstracts and conference proceedings, non-human studies, and non-English publications were also excluded.

Outcomes of interest and data abstraction
The primary outcome of this study was to characterize the existing adaptive surgical trial literature and assess the methodological and reporting quality.All identified citations underwent screening of titles and abstracts in duplicate (E.O. and S.L.), followed by full-text evaluation in duplicate (P.S. and E.O.).With the use of a standardised and piloted data abstraction template, the following study characteristics were extracted: publication year, country of study, study design, and methodological details pertaining to adaptive design.We used CONSORT ACE 2020 to guide data abstraction and identified the following adaptive trial characteristics: type of adaptive design, number and type of pre-determined interim analyses, goals of interim analysis, presence of any statistical simulations, and details related to randomization, blinding, type I error adjustments, and final statistical analysis [17].All data abstraction was performed in duplicate (E.O. and S.L.) and any conflicts resolved by third reviewer (P.S.).

Quality appraisal
For all studies meeting eligibility criteria, we performed quality appraisal using the Cochrane Risk-of-Bias for Randomized Trials (RoB 2.0) instrument [18].We also performed a reporting quality appraisal using the CONSORT ACE 2020 guidelines for adaptive trials [17].The author checklist was applied to the abstract and main text for all included articles.We categorized quality of reporting into fully reported, partially reported (with details provided, where relevant), and not reported.All quality appraisal was performed in duplicate (E.O. and S.L.).All conflicts were resolved via discussion and a third reviewer (P.S.).

Statistical analysis
We performed descriptive statistical analysis for all included studies.We reported all continuous outcomes as means (with standard deviation) or median (with ranges), where applicable.All categorical outcomes were reported as proportions and percentages, where applicable.All analyses were performed in Microsoft Excel (Redmond, Washington, USA).

Search strategy and article selection
Our database search yielded 1338 results from database inception to October 2023, of which, six published trials met eligibility criteria (i.e., <0.5% of retrieved citations) (Fig 1) [19][20][21][22][23][24].Our review of clinicaltrials.govyielded 263 active trials, but none met the eligibility criteria.Our review of pre-publication databases did not yield any manuscripts that met eligibility criteria.

Study characteristics
All six included studies were published after 2015 (Table 2) [19][20][21][22][23][24].They were in the fields of cardiothoracic surgery, orthopedics, urology, and oral and maxillofacial surgery.Five studies published their study protocol in advance of trial registration [19][20][21][22][23].All studies compared two treatment arms.Two trials were conducted in the US (33.3%), one in Japan (16.7%), and three in Europe (50%).Two trials were multi-institutional [21,23].The total number of recruited patients was 2980 and the range of patient recruitment was 30-1746 patients.We also note that three trials recruited under 150 patients [20][21][22].All studies utilized at least one surgical treatment arm.One cardiothoracic surgery trial utilized an interventional procedure as an experimental arm [23].All studies utilized 1:1 randomization, and blinding included a  combination of patients, outcome assessors, and/or surgeons.These studies did not report an expertise-based group allocation design.Primary outcomes in five trials clinical endpoints of direct patient relevance [19][20][21][22][23].In the five studies that reported secondary outcomes, the majority were further clinical endpoints with two studies evaluating patient-centred outcomes, and one performing a global economic analysis.The study follow-up range for all trials was 1-24 months with most studies using a 6-12-month study follow-up period.All study data is included in S2 File.

Adaptive methodology characteristics
All included studies were prospective adaptive RCTs (Table 3).One study specified a non-inferiority design [23].Two studies employed adaptive randomization [20,24].Mastroianni 2017) mentioned the role of statistical simulations in informing their trial design and analysis plan [21,23].Four studies described their proposed statistical analysis plan, including the type of group allocation analysis and the relevant inferential testing methods.Reardon et al. ( 2017) was the only study to employ a Bayesian method for trial design and analysis, and this group also used a statistical consulting firm to assist with calculations [23].Three studies described a group-sequential design with two treatment arms alongside sample size re-estimation performed at each interim analysis.Four studies also described their adaptive methodologies, including the number and goal(s) of their interim analyses [19,[21][22][23].These interim analyses were all pre-specified and performed for the purpose of study termination for futility or efficacy, and/or sample size reestimation.In two studies, the sample size was increased at the interim analysis [19,22].Two studies utilized two pre-specified interim analyses [19,21].Gaudino

Risk of bias and CONSORT reporting
All trials were evaluated by two independent reviewers in duplicate using the Cochrane RoB 2.0 instrument (Fig 2).Overall, three (50%) trials had a low risk of bias and three (50%) had a high risk of bias.A high risk of bias was primarily derived from the randomization process, deviations from intended interventions, and the reporting of outcomes [20,23,24].We also performed an assessment of reporting quality using the CONSORT extension ACE checklist [17].Two studies directly referenced CONSORT ACE 2020 guidelines [20,22].Most studies (83.3%) reported trial registration, protocol, full statistical analysis plan, and funding sources.We described abstract and main text reporting outcomes for all included studies (S1 Table ; S3 File).Despite other studies adequately reporting details for rationale, methods, and results, one study reported adaptive design details within their abstract [21].We found that within the methods section of the main text, most reporting heterogeneity (i.e., 50-83.3% of studies describing either partial or no reporting) occurred when describing design changes following trial initiation, changes to study design after the start of the trial, and details regarding blinding implementation and adherence (Fig 3).In the results section, no studies reported the reasons for trial stoppage or sufficient details surrounding this decision, and there was heterogeneity in the quality of outcome reporting.Lastly, regarding the discussion, we found that findings were adequately contextualized, but only two studies fully reported study limitations with direct reference to adaptive design decisions [21,22].

Discussion
We present the first scoping review of adaptive clinical trials in surgery.Since 2015, six adaptive trials have been published comparing at least one surgical intervention, which only represents 0.5% of the database search.We also note that three trials recruited under 150 patients.Adaptive methodologies are increasingly being adopted in non-surgical trials, but their design remains nebulous and as such, they have seldom been applied to surgical populations [13].
Adaptive randomization methods represent an early adaptive method applied to trial design in medicine [25,26].Response-adaptive randomization can minimize the number of patients randomized to an inferior treatment arm and may suit multi-arm trials or trials with prolonged recruitment and moderately delayed responses, [27,28].This method of randomization may improve surgical trial recruitment and ethicality, as patients will have an increased probability of being randomized to an effective surgical arm [28].Sirkis et al. (2022) simulated adaptive randomization techniques in the RECOVERY trial demonstrating that adaptive approaches may have reduced death and increased the likelihood of randomization to effective COVID-19 therapies [29].Adaptive randomization has the potential, however, to optimize patient recruitment in surgical trials, but must be used prudently, as surgical trials are often small-to-moderate in size and therefore prone to prognostic imbalances.A priori trial simulation studies to guide sample size calculations and evaluate optimal adaptive randomization methods and their biases may further inform adaptive designs for surgical trials.
Outside of adaptive randomization alone, we found that three studies employed a group sequential design with blinded sample size re-estimation performed at one or two prespecified interim analyses [19,21,22].A recent systematic review identified 27 adaptive trials that utilized sample size re-estimation, in particular, this design was used in Phase I/II trials wherein there was insufficient prior knowledge of treatment group [30].Sample size re-estimation, however, can lead to an inflated type I error rate, and though methods for optimizing these statistical adaptations exist, they remain poorly standardized within the adaptive trial literature [31].Group sequential trial designs are defined by at least one interim analysis being built into the trial design to evaluate efficacy and/or futility of treatment arms and can facilitate the adding or dropping of additional treatment arms [32].Sequential designs have been used in pharmaceutical trials for decades, and more recently were employed in the DEVELOP-UK trial evaluating lung perfusion following lung transplantation [32,33].Two studies in this review reported that statistical simulations were performed to guide design and interim analyses [21,23].Statistical simulations represent another advantage of adaptive trials, as they can estimate efficiency gains and facilitate trial improvements prior to funding expenditure and beginning patient recruitment [34].Reardon et al. (2017) utilized a Bayesian analysis plan to design and perform analysis within their trial [23].Though statistically more complicated, Bayesiandesigned trials hold promise in comparative effectiveness trials wherein prior population knowledge is lacking, as these designs can assist sample size estimations, improve power, and potentially increase the rate at which patients are randomized to effective treatments [35].Current barriers to adaptive trial adoption amongst stakeholders, however, include patient consent, risks for bias, type I error rate, lack of clear rationale, and a paucity of education about adaptive methodologies [36].
Interestingly, combining adaptive methods such as sample size re-estimation couched within a response-adaptive RCT, may also prove helpful as the synergized application of these methods can further improve power and shorten trial duration [37].Reardon et al. (2017) reported that an independent statistical group assisted with their analysis [23].Better adoption of complex adaptive trial methodologies amongst surgeons will need to occur alongside improved biostatistical training for surgeons and increased collaboration with statisticians [38,39].Moreover, it is not difficult to conceptualize the interaction that can exist between generative artificial intelligence and the computations that underly adaptive trial design and execution [40].
We performed a quality appraisal using the Cochrane Rob 2.0 instrument for RCTs [18].We were unable to identify a quality appraisal tool tailored to adaptive trial methodologies, but our research group is currently in the process of creating a customized adaptive trial RoB instrument.Our quality appraisal identified 50% of the included trials to have a high risk of bias, which was primarily derived from randomization details and inadequate outcome reporting.We used the CONSORT ACE 2020 guidelines to assess reporting quality in all included trials [17].Within the main text, heterogenous reporting was primarily identified in the description of trial design, changes following the start of the trial, and details regarding blinding and concealment maintenance.These deficiencies are important to highlight, as adherence to pre-specified trial design parameters and interim analyses are critical in minimizing type I error rate and bias within adaptive methodologies.It will be important for future adaptive trials to adhere to the published CONSORT ACE 2020 guidelines.Our study limitations are the exclusion of any abstracts or grey literature, and despite liberal search criteria, the risk of missing any relevant surgical trials.
Surgical subspecialties are beginning to explore the role for adaptive trials in their respective disciplines [41,42].As we peer into the future of evidence-based surgery, we must identify the existing barriers to adoption and global dissemination of surgical trial design and implementation.For instance, our group is currently exploring the importance of pilot trials in surgery to overcome issues of early trial termination [5,43].As researchers characterize the importance of creativity in surgical innovation, we posit that adaptive trial methodologies provide yet another methodological tool to answer translational questions and advance evidence-based surgical knowledge [44].Since conventional RCTs are often conducted out of high-income countries, cost-effective and efficient adaptive methodologies may facilitate practice-changing trials being conducted in low-middle income countries, thereby improving global collaboration and conclusion generalizability [45].
Adaptive designs have the potential to optimize patient recruitment and statistical power when sample sizes are small, or little is known about the research populations under investigation: common issues encountered in surgical trials [46].These designs may also assist with conducting trials in rare disease populations [47].Like any technological advancement, however, it is imperative that we always identify the appropriate research question and clinical scenario where adaptive methodologies can be best deployed, as they are not without their shortcomings [48].For instance, adaptive trials are likely not best suited when there is a notable delay in outcome assessment or there is inadequate trial infrastructure at the primary institution.An adaptive design is, however, likely best suited when the gain in trial efficiency and cost-effectiveness greatly outweighs the added complexity to trial methodology and statistical analysis [48].Here, we demonstrate that the number of published adaptive surgical trials is low and reporting of complex adaptive methods is often heterogenous and inadequate.Future adaptive trials must be reported in accordance with published CONSORT ACE 2020 guidelines [17].As these methodologies continue to be optimized, we suggest that surgical trialists consider implementing adaptive design components when deemed appropriate for their clinical question and population-of-interest.Adaptive trial designs may help to improve the quality of surgical evidence, streamline time to reporting, and compliment the accelerated pace of innovation in surgery.

Table 2 . Summary of included studies. Author (year) Study design Country Surgical field No. of centres No. patients recruited No. patients analysed Treatment groups Primary outcome Secondary outcomes Type of adaptive design Blinding Data analysis plan Type of randomization Study follow- up Trial adjustments made at interim analysis Published protocol Experimental Control
-related quality of life; ITT, intention-to-treat; mITT, modified intention-to-treat; NR, not reported; RCT, randomized clinical trial; TAVR, trancatheter aortic valve replacement; TP-RARP, Transperitoneal robotic assisted radical prostatectomy https://doi.org/10.1371/journal.pone.0299494.t002